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DETAILED ACTION 
Response to Amendment 

1. The amendment filed on 05/06/2004 has been entered. Claims 1, 6, and 11, and 26-28 
have been amended. Claim 29 has been newly added. Claims 3-4; 9-10, and 14-25 have been 
canceled. Claims 1-2, 5-8, 11-13, 26-29 are pending in the application. 



Claim Rejections - 35 USC§112 

2. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

3. Claims 1-2, 5-8, 11-13, 26-29 are rejected under 35 U.S.C. 1 12, first paragraph, as failing 
to comply with the enablement requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to which it pertains, 
or with which it is most nearly connected, to make and/or use the invention. 

Claim 1 recites "a plane mapping mode" which is not enabled by the specification 
because the specification instead describes "a planar mapping mode". Claim 2 and 5 depend 
upon the claim 1 and are rejected due to their dependency on the claim 1 . 

Claim 6 recites "a plane mapping mode" which is not enabled by the specification 
because the specification instead describes "a planar mapping mode". Claim 7-8 depend upon the 
claim 6 and are rejected due to their dependency on the claim 6. 
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Claim 1 1 recites "a plane mapping mode" which is not enabled by the specification 
because the specification instead describes "a planar mapping mode". Claim 12-13 depend upon 
the claim 1 1 and are rejected due to their dependency on the claim 11. 

Each of the claims 26-29 recites "a plane mapping mode" which is not enabled by the 
specification because the specification instead describes "a planar mapping mode". 



4. The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claims 1-2, 5-8, 11-13, 26-29 are rejected under 35 U.S.C. 1 12, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

The claim 1 recites "a plane mapping mode" which is ambiguous because "a plane 
mapping mode" is not found in the specification. The page 41 of the specification recites "a 
planar mapping mode" which itself is not clearly defined. Claim 2 and 5 depend upon the claim 1 
and are rejected due to their dependency on the claim 1 . 

The claim 6 recites "a plane mapping mode" which is ambiguous because "a plane 
mapping mode" is not found in the specification. Claim 7-8 depend upon the claim 6 and is 
rejected due to their dependency on the claim 6. 

The claim 1 1 recites "a plane mapping mode" which is ambiguous because "a plane 
mapping mode" is not found in the specification. Claim 12-13 depend upon the claim 1 1 and is 
rejected due to their dependency on the claim 11, 
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Each of the claims 26-29 recites "a plane mapping mode" which is ambiguous because "a 
plane mapping mode" is not found in the specification. 



Claim Rejections - 35 USC§103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1-2, 5-8, 1 1-13, and 26-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Xiong et al. U.S. Patent No. 6,434,265 (hereinafter Xiong), and further in view 
of Shum et al. U.S. Patent No. 6,271,855 (hereinafter Shum) and Teo U.S. Patent No. 6,246,413 
(Teo). 

8. Claim 1: 

(a) Xiong teaches an image synthesis method comprising: 

An input step, of inputting a plurality of image data (e.g., column 9, lines 20-30); 

A placement information generating step, of generating placement information 

determined by a placement order of all images inputted in the input step (e.g., column 16, lines 

35-45); 

A placement information obtaining step of obtaining placement information about a 
plurality of images in which adjacent images have a common subject region (e.g., in column 4, 
lines 5-40, Xiong teaches a method for constructing a panorama from rectilinear images in 3D 
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through projective registration and calibration including: (1) the projective registrations of 
overlapping images, (2) calibration and global optimization of these images, a self calibration in 
which 2D image planes are positioned as 3D planes in space); 

A setting step of setting one mapping mode out of a plurality of mapping modes each 
corresponding to a different mapping surface in accordance with the obtained placement 
information (e.g., in column 8, lines 18-58 of Xiong, it is stated that "overlapping photographs 
are analyzed to determine what orientation the photographs were taken in order to establish a 
common ground for subsequent operations and the panorama is constructed on a particular 
geometry that will best facilitate the rendering of the projection of the panorama onto a chosen 
viewing plane for viewing". The Xiong discloses some typical geometry on which panoramas are 
formed; In column 8, lines 18-58, Xiong further discloses that panorama is constructed on a 
particular geometry that will best facilitate the rendering of the projection onto a chosen viewing 
plane such as cubic, polyhedral, cylindrical and spherical geometries); and 

A synthesis step of combining the plurality of images by using the mapping mode set in 
the setting step (e.g., in column 4, lines 5-40 of Xiong, it is stated that the composing or blending 
in which images are ready to be re-projected to a 3D environment map with pixels in overlap 
regions being composed from multiple; In column 8, lines 18-58, Xiong further teaches that 
overlapping photographs are analyzed to determine what orientation the photographs were taken 
in order to establish a common ground for subsequent operations and the panorama is 
constructed on a particular geometry that will best facilitate the rendering of the projection of the 
panorama onto a chosen viewing plane for viewing. The Xiong further discloses some typical 
geometry on which panoramas are formed); 
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A changing step, of changing the mapping mode (In column 4, Hnes 40-50, it is stated 
that "the projection module may be controlled through the user interface 230 as well, to allow a 
user to select what geometry will be projected onto". Therefore, Xiong teaches a changing step 
through the user interface 230 by selecting a geometric surface out of a plurality of geometric 
surfaces each corresponding to a different mapping surface). 



- The Examiner interprets "a placement information obtaining step of obtaining placement 
information about a plurality of images in which adjacent images have a common subject 
region" as an automatic registration and calibration step of registering the overlapping 
images and capturing common overlapping areas between overlapping images and 
minimizing the average squared pixel intensity difference with respect to certain 
transformation parameters. 

- The Examiner interprets the mapping mode as mapping images onto a geometric surface 
such as a planar or a cylindrical surface (Applicant's specification, page 1, lines 20-25). 
Accordingly, the Examiner interprets the setting step of setting a mapping mode as the 
selecting step of selecting a geometric surface. In column 8, lines 18-58, Xiong discloses 
that panorama is constructed on a particular geometry that will best facilitate the 
rendering of the projection onto a chosen viewing plane (a geometric surface) such as 
cubic, polyhedral, cylindrical and spherical geometries. In column 4, lines 40-50, it is 
stated that "the projection module may be controlled through the user interface 230 as 
well, to allow a user to select what geometry will be projected onto". Therefore, Xiong 
teaches a selecting step of selecting a geometric surface out of a plurality of geometric 
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surfaces each corresponding to a different mapping surface in accordance with the 
obtained placement information. As applied to the present application, Xiong fulfills the 
claimed limitation of a setting step of setting one mapping mode out of a plurality of 
mapping modes each corresponding to a different mapping surface in accordance with 
said obtained placement information. 

(b) However, Xiong is silent to "a setting step, of automatically setting one mapping 
mode out of a plurality of mapping modes each corresponding to a different mapping surface in 
accordance with the obtained placement information." 

(c) Teo teaches the claim limitation of 1) a setting step, of automatically setting one 
mapping mode out of a plurality of mapping modes each corresponding to a different mapping 
surface in accordance with the obtained placement information (Teo teaches manipulating 
(adjusting the scale of the polyhedral surface, rotating the surface, re-positioning edges of the 
polyhedral surfaces) a polyhedral surface upon which the scene is to be projected, relative to the 
initial panoramic image, to form a desired surface, wherein the desired surface is distinct from 
the initial surface and modifying the initial panoramic image by mapping points on the desired 
surface to corresponding points on the initial surface to produce a modified panoramic image 
corresponding to projection of the scene onto the desired surface. Teo therefore teaches 
automatically setting a mapping mode by the computer through a user interface); 

(d) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Teo's setting step because Xiong also teaches a user interface for setting a mapping surface by 
the computer and therefore Xiong suggests automatically setting a mapping mode. 
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(e) One having the ordinary skill in the art would have been motivated to do this because 
setting a mapping mode out from a plurality of mapping mode serves for the purpose of 
eliminating visible artifacts (Teo column 10). 

(f) However, Xiong and Teo are silent to issuing warning wherein the warning being 
issued in a case in which the synthesized image exceeds a predetermined angle of view when a 
cylindrical mapping mode is changed to a plane mapping mode in the claim limitation of "a 
generating step, of issuing , when an image formed by changing the mapping mode in the changing 
step does not comply with a predetermined condition set in accordance with the mapping mode, a 
warning and generating a synthesized image in accordance with the predetermined condition, the 
warning being issued in a case in which the synthesized image exceeds a predetermined angle of 
view when a cylindrical mapping mode is changed to a plane mapping mode" (i.e., generation of a 
warning message). 

(g) Shum implicitly teaches issuing warning wherein the warning being issued in a case 
in which the synthesized image exceeds a predetermined angle of view when a cylindrical 
mapping mode is changed to a plane mapping mode in the claim limitation of "a generating step, 
of issuing , when an image formed by changing the mapping mode in the changing step does not 
comply with a predetermined condition set in accordance with the mapping mode, a warning and 
generating a synthesized image in accordance with the predetermined condition, the warning 
being issued in a case in which the synthesized image exceeds a predetermined angle of view 
when a cylindrical mapping mode is changed to a plane mapping mode" (e.g., Shum teaches 
determining the constraints governing the mapping mode of the mapping onto the projection 
plane wherein the constraints incorporates the predetermined angle of view or the predetermined 
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camera orientation matrix by the governing linear systems of equations. Shum further teaches 
generating a warning message to indicate that the constraint equations are unsolvable in a case in 
which the synthesized image is not aligning with any portion of the projected model when the 
projection plane is changed or when the angle of view is changed, e.g., the projection plane is 
changed from a cylindrical plane to a spherical plane or the angle of view has changed. When the 
linear systems of equations are not solvable, a user can switch the mapping mode to meet the 
constraint equations requirements by changing the plane normals. Moreover, the projection space 
can be in the form of any shape such as spherical or cylindrical and by changing the camera's 
point of view and the coordinates and directions of the plane normals, the projection space is 
effectively changed and thereby the mapping mode is changed. See column 13-22). 

(h) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Shum's warning message generation step because Xiong suggests a generating step of generating 
a synthesized image in accordance with the predetermined condition (Xiong column 3, lines 35- 
55; column 17, lines 15-67; column 18, lines 1-4). Moreover, Xiong teaches in figures 2 and 3 a 
user interface and a global optimization that provides feedback to the computer system such as 
issuing warning messages on the computer monitor 218 when the pair-wise objective function is 
not desirable for a poor selection of the projection viewing plane (and the resulting panoramas 
will have imperfectly aligned images that give shadow or ghosting effects) due to a poor 
selection of the projection viewing plane and the resulting panoramas will have imperfectly 
aligned images that give shadow or ghosting effects. Xiong also points to a seamless multi- 
resolution average blending method that would result in an absent of shadow effects (column 14, 
lines 1-45). Xiong further teaches a human interaction being present throughout the image 
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synthesis process to provide feedback to the computer system in all the nonlinear optimizations 
to let users monitor the progress of the system and allow them to intervene when necessary and 
therefore the claimed limitation suggests an obvious modification of Xiong because providing 
feedback to users is similar to issuing a message to users in a user interaction with the nonlinear 
optimizations when the changing step does not comply with a predetermined condition set or 
user-selected parameter set. 

(i) One having the ordinary skill in the art would have been motivated to do this because 
this would have provided the user the informative message such as issuing a warning message 
through the user interaction when convergence to the predetermined condition setting cannot be 
obtained in the image synthesis (Xiong column 17, lines 15-67, column 18, lines 1-4) for non- 
solvable optimization problems (Shum column 18, lines 38-67). 

Claim 2: 

Claim 2 recites all the limitations of claim 1 and adds the limitation of "a focal length 
obtaining step." Xiong teaches finding projective parameters such as 3D rotation parameters 
(pan, tilt roll), center of projection of images, ratio of focal lengths, and the like (column 10, 
lines 9-28). Xiong implicitly teaches finding the camera internal parameters vector of which the 
focal length is a component (column 1 1, lines 15-42), 

Claim 5: 

The claim 5 recites all the limitations of claim 1 and adds the limitation of "a displaying 
step of displaying a cuttable rectangular region." Xiong teaches how to align images more 
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precisely by changing the coordinates for positioning an image. Xiong further teaches placing the 
images 1210 at selected tangents on the viewing sphere 1220 (figure 12, and column 17, lines 
12-65). 

9. Claim 6: 

The claim 6 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of "an image synthesis apparatus" and "a displaying step of displaying a 
cuttable rectangular region". However, Xiong further discloses in figure 3 an apparatus for image 
synthesis. Xiong teaches how to align images more precisely by changing the coordinates for 
positioning an image. Xiong further teaches placing the images 1210 at selected tangents on the 
viewing sphere 1220 (figure 12, and column 17, lines 12-65). 

Claim 7: 

The claim 7 recites all the limitations of claim 6 and adds the limitation of "a focal 
length obtaining step." The Xiong reference teaches finding projective parameters such as 3D 
rotation parameters (pan, tilt roll), center of projection of images, ratio of focal lengths, and the 
like (column 10, lines 9-28). Xiong implicitly teaches finding the camera internal parameters 
vector in his image synthesis apparatus of figure 3 because the focal length is a component of 
that vector (column 11, lines 15-42). 

Claim 8: 

The claim 8 recites all the limitations of claim 6 and adds the limitation of "a changing 
step of changing the mapping mode." Xiong teaches that panorama is constructed on a particular 
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geometry that will best facilitate the rendering of the projection of the panorama onto a chosen 
viewing plane such as cubic, polyhedral, cylindrical and spherical geometries (column 8, lines 
18-58). 



10. Claim 11: 

The claim 1 1 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of a computer-readable storage medium having a program for 
implementing image synthesis method. However, Xiong further discloses the claimed limitation 
of a computer-readable storage medium having a program for implementing image synthesis 
method (e.g., in column 3, lines 54-57, it is stated "a program residing in system memory 220 
which stores output data and other data"). 

Claim 12: 

The claim 12 recites all the limitations of claim 1 1 and adds "a focal length obtaining 
step." Xiong teaches finding projective parameters such as 3D rotation parameters (pan, tilt roll), 
center of projection of images, ratio of focal lengths, and the like (column 10, lines 9-28). Xiong 
implicitly teaches finding the camera internal parameters vector of which the focal length is a 
component (column 11, lines 15-42). 

Claim 13: 

The claim 13 recites all the limitations of claim 1 1 and adds the limitation of "a changing 
step of changing the mapping mode." Xiong teaches that panorama is constructed on a particular 
geometry that will best facilitate the rendering of the projection of the panorama onto a chosen 
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viewing plane such as one of the cubic, polyhedral, cylindrical and spherical geometries (column 
8, lines 18-58). 

11. Claim 26: 

(a) Xiong teaches an image synthesis method comprising: 

An input step, of inputting a plurality of image data (e.g., column 9, lines 20-30); 

A placement information generating step, of generating placement information 

determined by a placement order of all images inputted in the input step (e.g., column 16, lines 

35-45); 

A placement information obtaining step of obtaining placement information about a 
plurality of images in which adjacent images have a common subject region (e.g., in column 4, 
lines 5-40, Xiong teaches a method for constructing a panorama from rectilinear images in 3D 
through projective registration and calibration including: (1) the projective registrations of 
overlapping images, (2) calibration and global optimization of these images, a self calibration in 
which 2D image planes are positioned as 3D planes in space); 

A setting step of setting one mapping mode out of a plurality of mapping modes each 
corresponding to a different mapping surface in accordance with the obtained placement 
information (e.g., in column 8, lines 18-58 of Xiong, it is stated that "overlapping photographs 
are analyzed to determine what orientation the photographs were taken in order to establish a 
common ground for subsequent operations and the panorama is constructed on a particular 
geometry that will best facilitate the rendering of the projection of the panorama onto a chosen 
viewing plane for viewing". The Xiong discloses some typical geometry on which panoramas are 
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formed; In column 8, lines 18-58, Xiong further discloses that panorama is constructed on a 
particular geometry that will best facilitate the rendering of the projection onto a chosen viewing 
plane such as cubic, polyhedral, cylindrical and spherical geometries); and 

A synthesis step of combining the plurality of images by using the mapping mode set in 
the setting step (e.g., in column 4, lines 5-40 of Xiong, it is stated that the composing or blending 
in which images are ready to be re-projected to a 3D environment map with pixels in overlap 
regions being composed from multiple; In column 8, lines 18-58, Xiong further teaches that 
overlapping photographs are analyzed to determine what orientation the photographs were taken 
in order to establish a common ground for subsequent operations and the panorama is 
constructed on a particular geometry that will best facilitate the rendering of the projection of the 
panorama onto a chosen viewing plane for viewing. The Xiong further discloses some typical 
geometry on which panoramas are formed); 

- The Examiner interprets "a placement information obtaining step of obtaining placement 
information about a plurality of images in which adjacent images have a common subject 
region" as an automatic registration and calibration step of registering the overlapping 
images and capturing common overlapping areas between overlapping images and 
minimizing the average squared pixel intensity difference with respect to certain 
transformation parameters. 

- The Examiner interprets the mapping mode as mapping images onto a geometric surface 
such as a planar or a cylindrical surface (Applicant's specification, page 1, lines 20-25). 
Accordingly, the Examiner interprets the setting step of setting a mapping mode as the 
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selecting step of selecting a geometric surface. In column 8, lines 18-58, Xiong discloses 
that panorama is constructed on a particular geometry that will best facilitate the 
rendering of the projection onto a chosen viewing plane (a geometric surface) such as 
cubic, polyhedral, cylindrical and spherical geometries. In column 4, lines 40-50, it is 
stated that "the projection module may be controlled through the user interface 230 as 
well, to allow a user to select what geometry will be projected onto". Therefore, Xiong 
teaches a selecting step of selecting a geometric surface out of a plurality of geometric 
surfaces each corresponding to a different mapping surface in accordance with the 
obtained placement information. As applied to the present application, Xiong fulfills the 
claimed limitation of a setting step of setting one mapping mode out of a plurality of 
mapping modes each corresponding to a different mapping surface in accordance with 
said obtained placement information. 

(b) However, Xiong does not implicitly teach "a setting step, of automatically setting one 
mapping mode out of a plurality of mapping modes each corresponding to a different mapping 
surface in accordance with the obtained placement information." 

(c) Teo teaches the claim limitation of a setting step, of automatically setting one 
mapping mode out of a plurality of mapping modes each corresponding to a different mapping 
surface in accordance with the obtained placement information (Teo teaches manipulating 
(adjusting the scale of the polyhedral surface, rotating the surface, re-positioning edges of the 
polyhedral surfaces) a polyhedral surface upon which the scene is to be projected, relative to the 
initial panoramic image, to form a desired surface, wherein the desired surface is distinct from 
the initial surface and modifying the initial panoramic image by mapping points on the desired 
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surface to corresponding points on the initial surface to produce a modified panoramic image 
corresponding to projection of the scene onto the desired surface. Teo therefore teaches 
automatically setting a mapping mode by the computer through a user interface); 

(d) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Teo's automatically setting step because Xiong teaches a user interface for setting a mapping 
surface by the computer automatically and therefore Xiong suggests automatically setting a 
mapping mode. 

(e) One having the ordinary skill in the art would have been motivated to do this because 
setting a mapping mode out from a plurality of mapping mode serves for the purpose of 
eliminating visible artifacts (Teo column 10). 

(f) However, Xiong and Teo are silent to issuing warning wherein the warning being 
issued in a case in which the synthesized image exceeds a predetermined angle of view when a 
cylindrical mapping mode is changed to a plane mapping mode. 

(g) Shum implicitly teaches issuing warning wherein the warning being issued in a case 
in which the synthesized image exceeds a predetermined angle of view when a cylindrical 
mapping mode is changed to a plane mapping mode (e.g., Shum teaches determining the 
constraints governing the mapping mode of the mapping onto the projection plane wherein the 
constraints incorporates the predetermined angle of view or the predetermined camera 
orientation matrix by the governing linear systems of equations. Shum further teaches generating 
a warning message to indicate that the constraint equations are unsolvable in a case in which the 
synthesized image is not aligning with any portion of the projected model when the projection 
plane is changed or when the angle of view is changed, e.g., the projection plane is changed from 
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a cylindrical plane to a spherical plane or the angle of view has changed. When the linear 
systems of equations are not solvable, a user can switch the mapping mode to meet the constraint 
equations requirements by changing the plane normals. Moreover, the projection space can be in 
the form of any shape such as spherical or cylindrical and by changing the camera's point of 
view and the coordinates and directions of the plane normals, the projection space is effectively 
changed and thereby the mapping mode is changed. See column 13-22). 

(h) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Shum's warning message generation step because Xiong suggests a generating step of generating 
a synthesized image in accordance with the predetermined condition (Xiong column 3, lines 35- 
55; column 17, lines 15-67; column 18, lines 1-4). Moreover, Xiong teaches in figures 2 and 3 a 
user interface and a global optimization that provides feedback to the computer system such as 
issuing warning messages on the computer monitor 218 when the pair- wise objective function is 
not desirable for a poor selection of the projection viewing plane (and the resulting panoramas 
will have imperfectly aligned images that give shadow or ghosting effects) due to a poor 
selection of the projection viewing plane and the resulting panoramas will have imperfectlv 
aligned images that give shadow or ghosting effects . Xiong also points to a seamless muhi- 
resolution average blending method that would resuh in an absent of shadow effects (column 14, 
lines 1-45). Xiong further teaches a human interaction being present throughout the image 
synthesis process to provide feedback to the computer system in all the nonlinear optimizations 
to let users monitor the progress of the system and allow them to intervene when necessary and 
therefore the claimed limitation suggests an obvious modification of Xiong because providing 
feedback to users is similar to issuing a message to users in a user interaction with the nonlinear 
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optimizations when the changing step does not comply with a predetermined condition set or 
user-selected parameter set. 

(i) One having the ordinary skill in the art would have been motivated to do this because 
this would have provided the user the informative message such as issuing a warning message 
through the user interaction when convergence to the predetermined condition setting cannot be 
obtained in the image synthesis (Xiong column 17, lines 15-67, column 18, lines 1-4) for non- 
solvable optimization problems (Shum column 18, lines 38-67). 

12. Claim 27: 

The claim 27 encompasses the same scope of invention as that of the claim 26 except 
additional claim limitation of "a displaying step of displaying a cuttable rectangular region," 
Xiong teaches how to align images more precisely by changing the coordinates for positioning 
an image. Xiong further teaches placing the images 1210 at selected tangents on the viewing 
sphere 1220 (figure 12, and column 17, lines 12-65). 

13. Claim 28: 

The claim 28 encompasses the same scope of invention as that of claim 26 except 
additional claimed limitation of a computer-readable storage medium having a program for 
implementing image synthesis method. However, Xiong further discloses the claimed limitation 
of a computer-readable storage medium having a program for implementing image synthesis 
method (e.g., in column 3, lines 54-57, it is stated "a program residing in system memory 220 
which stores output data and other data"). 
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Claim 29: 

The claim 29 encompasses the same scope of invention as that of the claim 1 . The claim 
29 is subject to the same rationale of rejection set forth in the claim 1 . 

Remarks 

14, Applicant's arguments, filed 05/06/2004, paper number 16, have been fully considered 
but they are not deemed to be persuasive. 

15. Applicant argues in essence with respect to the amended claim 1 and similar claims that: 
(A) "Applicants point out that nothing has been found, or pointed out, in Shum that 
would teach or suggest anything about changing a mapping mode at all. ..The Examiner's 
assertion, therefore, appears to be totally incorrect, unless the Examiner's view is that it is 
somehow 'implicit' in that patent that selection of mapping modes is being discussed, and 
that that patent somehow 'implicitly' is really concerned with problems encountered 
upon change of mapping mode. Since such seems self-evidently not to be the case, the 
rejection is believed to be improper, and its reconsideration and withdrawal are 
respectfully requested." 

In response to the arguments in (A), the Examiner asserts that Shum implicitly teaches 
issuing warning wherein the warning being issued in a case in which the synthesized image 
exceeds a predetermined angle of view when a cylindrical mapping mode is changed to a plane 
mapping. For example, Shum teaches determining the constraints governing the mapping mode 
of the mapping onto the projection plane wherein the constraints incorporates the predetermined 
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angle of view or the predetermined camera orientation matrix by the governing linear systems of 
equations. Shum further teaches generating a warning message to indicate that the constraint 
equations are unsolvable in a case in which the synthesized image is not aligning with any 
portion of the projected model when the projection plane is changed or when the angle of view is 
changed, e.g., the projection plane is changed from a cylindrical plane to a spherical plane or the 
angle of view has changed. When the linear systems of equations are not solvable, a user can 
switch the mapping mode to meet the constraint equations requirements by changing the plane 
normals. Moreover, the projection space can be in the form of any shape such as spherical or 
cylindrical and by changing the camera's point of view and the coordinates and directions of the 
plane normals, the projection space is effectively changed and thereby the mapping mode is 
changed (column 13-22). 

It would have been obvious to one of ordinary skill in the art to have incorporated the 
Shum's warning message generation step because Xiong suggests a generating step of generating 
a synthesized image in accordance with the predetermined condition (Xiong column 3, lines 35- 
55; column 17, lines 15-67; column 18, lines 1-4). Moreover, Xiong teaches in figures 2 and 3 a 
user interface and a global optimization that provides feedback to the computer system such as 
issuing warning messages on the computer monitor 218 when the pair-wise objective function is 
not desirable for a poor selection of the projection viewing plane (and the resulting panoramas 
will have imperfectly aligned images that give shadow or ghosting effects) due to a poor 
selection of the projection viewing plane and the resulting panoramas will have imperfectly 
aligned images that give shadow or ghosting effects . Xiong also points to a seamless multi- 
resolution average blending method that would result in an absent of shadow effects (column 14, 
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lines 1-45). Xiong further teaches a human interaction being present throughout the image 
synthesis process to provide feedback to the computer system in all the nonlinear optimizations 
to let users monitor the progress of the system and allow them to intervene when necessary and 
therefore the claimed limitation suggests an obvious modification of Xiong because providing 
feedback to users is similar to issuing a message to users in a user interaction with the nonlinear 
optimizations when the changing step does not comply with a predetermined condition set or 
user-selected parameter set. 

One having the ordinary skill in the art would have been motivated to do this because 
this would have provided the user the informative message such as issuing a warning message 
through the user interaction when convergence to the predetermined condition setting cannot be 
obtained in the image synthesis (Xiong column 17, lines 15-67, column 18, lines 1-4) for non- 
solvable optimization problems (Shum column 18, lines 38-67). 



16. Applicant argues in essence with respect to the amended claim 1 and similar claims that: 
(B) "Moreover, considering only what is actually found in Shum '412, the Office Action 
fails to provide any reason why one of only ordinary skill, provided with the Shum use of 
a warning where a constraint encountered in the solution of a 3D modeling problem is 
left unmet, would somehow be motivated to issue a warning upon a certain condition 
occurring as a result of a change in mapping mode." 
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In response to the arguments in (B), the Examiner asserts that Shum implicitly teaches 
issuing warning wherein the warning being issued in a case in which the synthesized image 
exceeds a predetermined angle of view when a mapping mode is changed. For example, Shum 
teaches determining the constraints governing the mapping mode of the mapping onto the 
projection plane wherein the constraints incorporates the predetermined angle of view or the 
predetermined camera orientation matrix by the governing linear systems of equations. Shum 
further teaches generating a warning message to indicate that the constraint equations are 
unsolvable in a case in which the synthesized image is not aligning with any portion of the 
projected model when the projection plane is changed or when the angle of view is changed, e.g., 
the projection plane is changed from a cylindrical plane to a spherical plane or the angle of view 
has changed. When the linear systems of equations are not solvable, a user can switch the 
mapping mode to meet the constraint equations requirements by changing the plane normals. 
Moreover, the projection space can be in the form of any shape such as spherical or cylindrical 
(since any wall, ceiling or floor of the room can be in any shape) and by changing the camera's 
point of view and the coordinates and directions of the plane normals, the projection space is 
effectively changed and thereby the mapping mode is changed (column 13-22). 

It would have been obvious to one of ordinary skill in the art to have incorporated the 
Shum's warning message generation step because Xiong suggests a generating step of generating 
a synthesized image in accordance with the predetermined condition (Xiong column 3, lines 35- 
55; column 17, lines 15-67; column 18, lines 1-4). Moreover, Xiong teaches in figures 2 and 3 a 
user interface and a global optimization that provides feedback to the computer system such as 
issuing warning messages on the computer monitor 218 when the pair- wise objective function is 
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not desirable for a poor selection of the projection viewing plane (and the resulting panoramas 
will have imperfectly aligned images that give shadow or ghosting effects) due to a poor 
selection of the projection viewing plane and the resulting panoramas will have imperfectly 
aligned images that give shadow or ghosting effects . Xiong also points to a seamless multi- 
resolution average blending method that would result in an absent of shadow effects (column 14, 
lines 1-45). Xiong further teaches a human interaction being present throughout the image 
synthesis process to provide feedback to the computer system in all the nonlinear optimizations 
to let users monitor the progress of the system and allow them to intervene when necessarv and 
therefore the claimed limitation suggests an obvious modification of Xiong because providing 
feedback to users is similar to issuing a message to users in a user interaction with the nonlinear 
optimizations when the changing step does not comply with a predetermined condition set or 
user-selected parameter set. 

One having the ordinary skill in the art would have been motivated to do this because 
this would have provided the user the informative message such as issuing a warning message 
through the user interaction when convergence to the predetermined condition setting cannot be 
obtained in the image synthesis (Xiong column 17, lines 15-67, column 18, lines 1-4) for non- 
solvable optimization problems (Shum column 18, lines 38-67). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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